Background subtraction is a common technique for detecting moving objects in a largely stationary environment. Generally, background subtraction techniques compare a current image against a reference “empty” image and note regions of change. Such differences generally correspond to the moving foreground objects of interest for applications such as surveillance. However, a background model must first be created and then maintained.
A background model can be created, for example, by memorizing an “empty” image where there are no visible objects. However, this method is not generally applicable because it is difficult to ensure that a scene is clear of all moving objects, especially if it is being remotely monitored. Another technique monitors an incoming video stream for motion energy by subtracting adjacent frames When the overall energy is low enough, an input frame is captured and thereafter used as the background model. Unfortunately, if a moving object, such as a pedestrian, momentarily pauses, a background image might be acquired which erroneously includes this temporarily immobile object.
Conversely, moving objects may enter a scene and then stop moving (e.g., a parked car). Similarly, if a person enters a room and then becomes immobile, such as taking a nap, the person persists as a detected object. In many cases, such objects would be better interpreted as a part of the “new” background. Moreover, if a person is present when the reference “empty” image is acquired, the person will be detected once he or she starts to move across the scene. However, even after the person has completely exited the scene, a “hole” where the person was originally positioned will likely continue to be perceived as an object
The background updates can be significant events in themselves. If an item is introduced into the scene (such as a briefcase carried surreptitiously by some agent), the item will also be marked as part of the foreground despite having no motion itself. A related situation is where some object that was part of the original reference image (e.g., a laptop computer) is removed from the scene. There will be a difference “hole” left behind in this case that is not only non-moving, but also not a solid object. There are several methods that can be used to locate non-moving regions in a background subtraction system, but it is difficult to classify such regions as abandoned objects, removed objects, state changes, or another phenomena.
The region type can be determined, for example, by comparing the pixel pattern in the region to the template of some known object (e.g., a computer monitor). If there is a non-moving foreground region and the associated pixels match the template, then a deposit event has occurred, otherwise a removal event is recorded. However, this approach requires that the system have a number of templates for each kind of object it cares about. Typically, such models must be manually entered, or an operator has to at least mark the boundaries of some region. A related approach requires that the system know something about the background near the non-moving region. If, for instance, the room had uniformly green walls, then the interior of the non-moving region could be examined to see if it was green or not (implying a removal or a deposit, respectively). Again, this is not a general purpose solution and works best if the characteristics of the environment can be chosen at will (like applying green paint).
Updating the background model is particularly important for environments in which lighting changes over time (e g, outdoors). Otherwise, lighting changes, such as the sun coming out from behind clouds, can cause large areas of the image to be falsely declared as foreground as they become better illuminated. Updating the background model is often done by slowly blending in newly acquired images with the old model. However, if the blending rate is fast and applied to the whole image, moving objects start to leave ghostly trails across the background model. Such anomalies can cause the system to both falsely detect nonexistent objects and miss detecting some valid objects. If, on the other hand, the blending rate is set very slow to reduce this effect, the system might not adapt quickly enough to the types of lighting changes that are present. Another updating option is to periodically reinitialize the background model from scratch using the current video frame This might be done, for instance, either on a regular schedule or when too large a portion of the image is marked as foreground While this approach prevents ghosting, it still suffers from the same initial model acquisition problems as described above
Background blending, as discussed above, will eventually “erase” all stationary objects as well as “holes” left by removed objects. Unfortunately, this method tends to erase objects by shrinking them, leaving invalid partial objects as the regions are absorbed into the background. Moreover, background blending can also leave “ghost” objects behind if, say, a moving person lingers too long in one area. There exist more sophisticated systems that model the intensity at each pixel as one of several Gaussian distributions. These systems have better immunity to the “ghosting” problem, but the decision to switch from one Gaussian model to another is typically made independently for each pixel. This leads to objects or holes “sparkling” out, with incorrect ragged objects detected during the transition
As apparent from the above-described deficiencies with conventional techniques for generating and updating a background model, a need exists for methods and apparatus for improved techniques for generating and updating a background model A further need exists for methods and apparatus for visual background subtraction that address each of the above-identified problems using one or more software preprocessing modules.